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survey was made in l^fu), ratims of oradua^e nroarams are Pased 

upon responses to questionnaires distributed +o taoulty members: 
resDondents vere as^ced to give th^ir opinion on graduate faculty 
quality usinu the classifications; Pi st inqu i shed , s^rona, Good, 
Adequate, Marginal, and not sufficient tor doctoral trainino. The 
respondent vas instructed not to rate more than 5 departments as 
nistinauished. This paper examines: (1) thn differences between tfr 

196u and 19f ^ surveys; (2) the chanqos in evaluations departments 
at the University of Chicano; (3) the nurber of torn -ranked 
departments in l^fu 19f^; arid (U) the changes in rankings Irtveen 

and 19f?* Comnarirons are also made has‘=d on the dc^partmepts 
ranked in tfo tor 10 and the top 1^. On ^he basis of this survc*v, 
^erReloy ^^nd Barvard were first, followed hv Yrlo^ Chicaoo, ctanto’-d^ 
and Princeton, 'the appendix discusses the problem of weighting fVe 
various discimlinrs, and th^ nossibilitv veichting the mador 
divisions: (humanities, social sciences, Physical sciences, and 

hioloqv) by +he popcentage of doctorates awarded in the ar^s and 
sciences in (A^) 
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A RATING OF GRADUATE PROGRAMS 
A Report of the American Council on Education 
An Ana.ly3is by D. Gale Johnson* 

The second in a series of surveys of the quality of graduate education was 

undertaken under the auspices of the American Council on Education. The most 
recent survey was undertaken In 19^9 while the first was for 1964. 

A Rating of Graduate Programs presents the resuUs of the second survey 
of the quality of graduate programs In the United States. The earlier survey 
was based on data collected In 1964 while the most recent survey was made In 
1969 * The ratings of graduate programs are based upon responses to question- 
naires distributed to faculty members; respondents were asked to give their 
opinion of the quality of the graduate faculty and could cla sify a given 
department as Distinguished, Strong, Good, Adequate, Marginal and not suf- 
ficient for doctoral training. The respondent was Instructed not to rate n>ore 
than five departments as Distinguished. 

There are a number of differences In the presentation of the results of 
the two surveys. In 1964 the number of arts and science disciplines surveyed 
was 25; In I9&9 the number of disciplines Included was 32, Major changes In 
disciplines were made In the humanities and biological sciences. In 1964 
there v^ere approximately the same number of disciplines surveyed In each of 
the pFaJor divisions of the arts and sciences. However, the 1969 survey pre- 
sents Information for 10 disciplines In the humanities and biological sciences, 

7 In the social sciences and 5 In the physical sciences. Psychology was 
categorized as a biological science In 1964 and as a social science In 1969. 

^ Q *D. Gale Johnson Is Proressor In the Depai’lnent of Economics 

'^FRIC University of Chicago. His analysis responds to widerpread 

interest of faculty in such university rankings. He enphaslzes^ 
.lowever, that rankings of this nature are imperfect. 
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A second important difference In the presentation of the ratings Is 
absence of the average departmental score by university department In 1969; 

In 1964 an average departmental score was derived from the ratings given by 
the respondents. A third important difference Is that the distinction between 
Distinguished and Strong was not made in 1869. The primary common element In 
the presentation of the results of the two surveys Is the ranking by univer- 
sities of those departments that received average scores of 3.0 or more. 

My purpose In preparing this note is primarily to give some perspectives 
on the changes In evaluations of graduate programs over the period from 1964 
to 1969 , with special attention to The University of Chicago.^ While It Is 
possible to make comparisons of the relative standings of the graduate pro- 
grams In the arts and sciences among the top un I vers 1 1 les--and some are 
presented later--the Indications of changes In the evaluations are probably 
subject to less error than are the relative rankings of the various universities. 



A. Ch ange s I n Evaluations of Depart- 
ments aY The University of Chicago 

Though departmental scores were not puhHshed, each university Included 
In the ratings was provided with the average scores for Its departments. Based 
on this Information as well as the published data, some of the changes In the 
evaluation of departments at the University of Chicago between 1964 and 19&9 
can be shown. 

Number of departments ranked a s Distinguished ; In the 1964 surv# ^ an 
average score for the quality of the graduate faculty of more than 4.0 re- 
sulted In a ranking of Distinguished. In 1964 there were seven departments 
ranked as Distinguished; In I 969 Nve additional departments were so ranked 
and each of the seven so ranked In 1964 retained that ranking. 




Nhe two surveys Include ratings of engineering departments but do not 

Include ratings of other professional schools. All of the comments In this 
note refer only to the arts and science disciplines. 
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Changes in departmental s c ores : Of the 25 departments that were In- 

cluded In both surveys, 13 departments received higher scores, 7 the same 
score and 4 had lower scores In the recent survey, 

Nu mber of departments in the top five and top ten : In 25 depart- 

ments were Included In the survey. At that time Chicago had 7 departments 
In the top five departments and 13 fn the top ten. In 1969 32 departments 
were Included In the survey end I of the Chicago departments v^ere lo the top 
five and 22 In the top ten. In both cases the Increase In number of top 
ranked departments was substantially greater than would have been expected 
due to the increase In the number of departments included fn the survey, 

B , Comparison of the Number of Top 

"Ranked DepartmentV,7T9fl 1 9 69 

One comparison that can be made from the data provided In the 196^ and 
1969 surveys Is of the number of departments at different universities that 
were In a given r^jlatlve position, such as the top 5 and top 10, The top five 
approximates the category of Distinguished departments as used In the 196*^ 
survey, while the top 10 Includes the Distinguished departments and the best 
of the departments that were formerly rated as Strong (with average scores 
from 3*0 to U,0). In the foliating tabulation universities are ranked by 
the number of departments In the top five departments; In the case of ties that 
Include the fifth or tenth ranked department, all depa rtn-»€nts were Included 
that are so tied, 

U Is clear That Berkeley and Harvard had a substantially la*'ger pro- 
portion of their departments ranked In the top five In both 196 ^ and I9&9 
than any of the other universities. I n 1969 Chica g o had the third JATOfi^t 
n umber, followed closely by Yale. Stanford and Princet on* Again for I 969 In 

O 




TABLE 1 



NUMBER OF HIGH RANKING DEPARTMENTS FOR SELECTED UNIVERSITIES 



Un 1 vers 1 ty 


Top 

19&9 


Number of 
Five 

19&^ 


Departments 

Top Ten'* 

1969 


1964 


University of California, 
Berkeley 


28 


23 


30 


24 


Harvard 


27 


22 


29 


24 


Chicago 


14 


7 


22 


12 


Yale 


13 


9 


22 


14 


Stanford 


12 


5 


21 


15 


Princeton 


12 


8 


17 


14 


Michigan 


10 


7 


19 


16 


MIT 


8 


3 


13 


7 


CALIFORNIA Inst, of Tech. 


8 


5 


9 


8 


Columbl a 


5 


6 


13 


15 


Wisconsin 


8 


7 


19 


17 


Come 1 1 


3 


2 


17 


9 


Pennsylvania 


7 


3 


10 


7 


Illinois 


It 


2 


8 


7 


New York Unlvcrsl ty 


1 


0 


3 


1 



On^/ seven Russian departments were Individually ranked. 




terms of the departments ranked In the top 10, Berkeley and Harvard le.id, 
though the distance between them and the next several universities Is not so 
great as In the case of the top five departments. 

Because of the difference In the number of departments Included In the 
1964 and 1969 surveys, a direct comparison of the change In the number of 
departments In the top five or top ten Is somewhat difficult to Interpret, As 
noted above, 25 departments were Included In 1964 and 32 In 1969* Only two of the 
high ranking universities (Chicago and Stanford) had an Increase of seven In 
the number of top five departments between 1964 and 1969; Berkeley 
and Harvard each had an increase of five, Chicago had the largest 
increase in the number of departments in the top ten--10--followed 
by Cornell with eight, Berkeley and Stanford with six, and Harvard 
with five. 

^ • ^han ^es In Rankings, 1964 to 1969 

Comparisons of the number of departments rated as among the top 5 or 
top 10 Implies no distinctions among the departments within each category. In 
the absence of average scores by department and university, It Is possible to 
differentiate and aggregate the data by assigning a score to each department 
based on Its actual ranking as first, second, third or any other rank. Two 
such comparisons, with an emphasis upon change between 1964 and 1969, have been 
made. One Is based upon the departments that were ranked from first *.o tenth 
and the othe^* upon departments ranked from first to fifteenth, Ir the first 
case a department that was ranked first was given a score of 10, a second 
rank a score of 9, and so on until the tenth ranked department Wiis given a 
score of one; departments ranked lower than tenth received no score. In the 
second case the first ranked department received a score of 15 and each de- 
partment ranked between second and fifteenth received scores of 1 4 down to 1. 
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The scores so derived can be summed either for each university or for the 
four broad areas within the arts and sciences (humanities, social sciences, 
biological sciences and physical sciences). 

There are two problems In the comparison of such total scores. One Is 
that the number of disciplines surveyed within the broad divisions of the arts 
and sciences Is a function of the extent to which specialization has resulted 
In the creation of separate fields for the Ph.D. Thus the physical sciences 
are represented by only five disciplines, whMe there are ten each In the 
humanities and biological sciences. Thus to total the scores for a university 
as a whole is. In effect, giving twice as much weight to the humanities than 
to the physical sciences. A general guide to the appropriate weights by 
division might be the distribution of doctorates awarded. In 1966 the approxi- 
mate number of doctorates awarded were: Physical sciences, 3|800; social 

sciences (Including history), 3i300; humanities, 1 ,800 and biological sciences 
(excluding agriculture), 2,300. Thus In terms of relative emphasis In 
graduate education, the physical and social sciences should have significantly 
greater weight than the humanities or biological sciences. Except for a cal“ 
culatlon Included In an appendix note, the calculations presented In Tables 
2 and 3 weight the four divisions equally. 

Comparisons Based on the Departments 
Ranked In the Top Ten and Top Fifteen 

In the next two tables the higher the total score, the higher Is the Im- 
plied ranking of a university's graduate faculty. In these two tables the 
average scores, based on equal welgln for each of the four divisions, are 
given for 19614 and I 969 , The average scores for each division for each Included 



O 

ERIC 



7 



TABLE 2 



AVERAGE SCORES BY UNI 
RANKED 


iVERSiTY FOR DEPARTMENTS BASED 
DEPARTMENTS, ISt'* and 196 9 


ON TOP TEN 


Ufil versl ty 


Average 

1969 


‘ Score 

1961* 


Change 
1961* to 
1969 


Ber keley 


7.63 


8.20 


-0.63 


Harvard 


7.'*8 


8.50 


-i.02 


Yale 


3.9'* 


'*.03 


-0.09 


ChiCacQ 


3.8b 


2.86 


+ 1 .00 


Stanford 


3.6«i 


F;.66 


+0. 98 


Princeton 




3.72 


-0.28 


Hicl)igan 


3.01 


2.1*7 


+0.54 


Wf s :ons1 n 


2.65 


3.10 


-0.1*5 


Columb i a 


1.82 


2.88 


-1.08 
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TABLE 3 

AVERAGE SCORES BY UNIVERSITY FOR DEPARTMENTS BASED ON TOP 
FIFTEEN RANKED DEPARTMENTS, ISt'* AND 1969 



University 


Average Score 
1969 


1961* 


Change 
196^ to 
1969 


Berkeley 


12.50 


13.19 


-0.69 


Harvard 


11.93 


13.22 


-1.29 


Yale 


7.93 


7.75 


+0.18 


Chicago 


7.59 


5.80 


+ 1.79 


Stanford 


7. '♦I 


6.06 


+ 1.35 


Princeton 


6.77 


6.1*1 


+0.36 


Wisconsin 


6.1*3 


7.33 


-0.90 


Michigan 


6.37 


6.01 


+0.36 


Columbia 


5.35 


6.18 


-0.83 
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university are give<: in tcbles at the end o1' this note.^ The change In average 
scores from 196^ to 1969 are also given. 

Perhaps a word Is In order concerning uie interpretation of the size of 
the average scorer. In the scores based rn the top ten ranked departments, If 
all departments in a given university were rc.nked first the score would be 10; 

If all departments were In the top ten and the average ranking were five the 
university score would be 6, For the scores based on the top fifteen depart- 
ments, a similar interpretation holds though the maximum score Is increased 
to 15 and the score fo* a university with an average ranking of five would be II. 

Tables 2 and 3 are based on the average ranks of derartrnents by university 
for the nine universities that ranked highest la 196^ based on average de- 
partmental scores. In 196^ the rankii gs based on average departmental scores 
were Harvard, Berkeley, Yale, Princeton, Chicago, Stanford, Wisconsin, Michigan 
and Columbia. 

The averages ot the scores basad on the rankings of the top ten depart- 
ments (Table 2) Indicate that six universities suffered declines, with the 
largest declines being for Harvard and Columbia. A decline of 1.00 Implies 
an average decline in departmental ranking of one position. Chicago and Stan- 
ford Increased their average departmental ranking by one, based on the 

top ten ranked departments. 

The averages of scores based on the rankings of the top fifteen depart- 
ments (Table 3) sho\«/ much the san^ picture. Four universities had a decline 
In average departir^enta 1 rankings, villh Harvard having the largest decline *?nd 
Wisconsin the second largest. Princeton had a small Increase in score betv/een 
1964 and 1969 based on the top fifteen departments In contrast to a snail de- 
cline when scores were based on the top ten departments . Chicago and Stanford 




The scores are averaged rather than totalled for two reasor^s: First, to 
avoid difference In scores due to the number of disciplines included In the two 
surveys, and, second, to facilitate the weighting of the divisions to arrive at 
an average for an entire university. 
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had substantial Increases in aver a ge scores, with Chicago having the larg est 
Increase when the top fifteen depa rt ments are Included and Stanford the 
largest Increase based on the top ten departments . 

Clearly the distance between the top two 
universities (Harvard and Berkeley) and the next four (Yale, Chicago, 
Stanford and Princeton) was diminished over the five year period. 




APPENDIX NOTE 



The authors of A Rating of Graduate Programs state (page 33): '"There 

still exist serious difficulties In determining how disciplines should be 
weighted. Should classics count as heavily as psychology or electrical 
engineering?'" There is no doubt that a serious problem exists. In the pre- 
vious material it has been assumed that there siiould be no distinctions In 
weighting among departments In the same division of the arts and sciences 
and that each of the divisions (humanities, social sciences, physical 
sciences and biology) should be, In turn, given equal weight. There are 
other reasonable weighting systems that could be used. One Is the distribu- 
tion of graduate students among disciplines as measured by the number of 
doctorates awarded In recent years. It would be possible to weight each 
department or discipline by the percentage of all doctorates In the arts 
and sciences awarded In a given year or period. The other possibility, and 
the one Illustrated here, is to weight the major divisions by the percentage 
of doctorates awarded In the arts and sciences in 19^6.^ 

Based on the 19^6 data on doctorates awarded approximately 3^ per cent 
of the degrees in arts and sciences were In the physical sciences, 30 per 
cent in the social sciences, 20 per cent In the humanities and l6 per cent 
In the biological sciences. In Appendix Tables I and 2 the average 

*The data are from the National Academy of Sciences, Doctorate Re- 
cip ients from Uni ted States Universities, 1958-1966 (Washington, 1967), PP« 
T5-TT! Doctorates In TgrrculTure and Tores try have been excluded from the 
hiological sciences and history has been included as a social science since 
It Is so designated In A Rating of Graduate Programs . Degrees In engineering 
are not Included. 
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departmental scores {based on rankings) for each division are weighted by 
these percentages to arrive at a university average score. As is evident 
from Appendix Tables the method of weighting used does make a difference. 
Appendix Table I is based on departments ranked In the top ten and is thus 
comparable to Table 2, except for the difference In weighting. Appendix 
Table 2 is based on the scores for the top fifteen departments and Is com- 
parable to Table 3 above. Appendix Tables 2 and 3 also Include the average 
departmental scores by division based on the rankings of the disciplines within 
each division for nine universities. These average departmental scores by 
division were used to derive the results in Tables 2 and 3. 

The major effect of the weighting of the divisions by the proportion 
of doctorates awarded In the arts and sciences was to change the rankings of 
Yale and Chicago in 19&9 for scores based on either ten or fifteen depart- 
ments. Chicago ranks third and Yale fourth when weighting by national 
doctorate production Is used. 
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APPENDIX TABLE 1 



Average Scores of 10 Top Ranked Departments by Divisions 
and by Universities with Divisions Weighted by Doctorates 
Awarded In 1966. 



Division 



A-1969 


Harvard 


Be rke ley 


Yale 


Chicago 


Stanf 'd 


Princ ton 


Wis. 


Mich. 


Co lum 


Humanities 


7.65 


7 . 45 


6. 50 


3.^5 


2.00 


5.60 


1.90 


3.25 


3.35 


S oc . Sc 1 . 


7.21 


7.79 


93 


6.50 


3.36 


2.14 


3 . 36 


6.21 


2.14 


Bio logy 


5.95 


6.80 


2.35 


1 . 70 


4.40 


0.50 


3.45 


2.40 


0.00 


Phy. Sci. 


8.60 


8.50 


2.00 


3. 80 


4.80 


5.50 


1.90 


0.20 


1.80 



Weighted 




















Average Score 


7.57 


7.80 


3.83 


4.20 


3.74 


3.71 


2 . 59 


2.97 


1.92 


^1964 _ _ 




















Humanl ties 


9. 50 


8. 34 


7.00 


1 .67 


0. 33 


6.25 


2.42 


2.25 


4.08 


Soc. Scl. 


8.14 


7.93 


5.21 


6, 35 


2.93 


2,43 


4,07 


4.00 


3. 78 


Biology 


7.57 


7.89 


1.71 


0.00 


2.86 


0,00 


4,14 


2.86 


0. 14 


Phy. Scl. 


8.80 


8,i0 


2.20 


3. 40 


4.00 


6.20 


1.70 


0. 60 


3.50 



Weighted 



Average Score 


8. 55 


8.23 


3.98 


3.40 


2.86 


4.09 


2.96 


2 .31 


3.16 


Difference 

1969-1964 


-0.98 


-0.43 


-0.15 


+ 0.80 


+ 0.88 


-0.38 


-0.37 


+ 0.66 


-1.34 




APPENDIX TABLE 2 



Average Scores of 15 Top Ranked Departments by Divisions 
and by Universities with Divisions Weighted by Doctorates 
Awarded in 1966 . 



Div j s ion 




















A-l<-69 


Harvard 


Be rkeley 


Yale 


Chicago 


Stanf 'd 


Princt ' 


n Wis. 


Mich. 


Co 1 u 


Hum?. n i t ies 


12,65 


12.45 


11.50 


7 . 25 


5. 35 


10. 10 


4.85 


7 . 40 


7.00 


Soc . Sc i . 


11.51 


12.81 


8.86 


10.80 


7,08 


4.93 


8.01 


11.23 


5.51 


b t o J. o g y 


9.95 


13 . 25 


5.45 


3. 70 


8.40 


1.95 


6.95 


4.85 


2.50 


Phy, Sci. 


IB, 60 


13. 50 


5.90 


8.60 


8.80 


10. 10 


5.90 


2 .00 


6 . 50 


Weighted Average 


















Score 

B-1S64 


12.20 


12.72 


7 . 84 


8.21 


7.53 


7.24 


6.49 


6 . 30 


5.66 


Duma n i t ies 


1^ , 50 


13.50 


11 , 50 


6. 17 


3.33 


9.42 


6 17 


6.25 


6.83 


Soc. Sci. 


12, 71 


13.06 


8,92 


10.64 


6. 78 


5 . 28 


9.35 


9 .00 


7.64 


Biology . 


11. 85 


i2. 71 


3,57 


0.70 


6,14 


0.35 


7.71 


5. 70 


1.86 


Phy. Sci. 


13.80 


13. 50 


7.00 


7.40 


8,00 


10.60 


6 . 10 


3.10 


8.40 



V7e i g h t e d 
Average Score 


13.30 


13.24 


7,93 


7.05 


6,40 


7.13 


7.35 


5 . 92 


6.81 


Difference 

1969-1964 


-1. 10 


-0.52 


-0.09 


+ 1 . 16 


+ 1, 13 


-»0. 11 


-0. j6 


+ 0.38 


-1.15 
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